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Executive Summary 

The Office of Shared Accountability (OSA) conducted an evaluation of the Linkages to Learning 
initiative in Montgomery County Public Schools (MCPS). Linkages to Learning (LTL) is a 
collaborative initiative among the Montgomery County Department of Health and Human Services 
(MCDHHS), MCPS, and local non-profit agencies. The goal of the program is to address the 
social, economic, health, and emotional issues that interfere with the academic success of a child 
(MCDHHS, 2015). LTL focuses its services on addressing three broad areas of need: 

• Student well-being. Services include assessment for social-emotional, behavioral 
concerns; consultation with teachers; child/family/group therapy; psychosocial skills 
development groups; primary care and treatment at LTL school-based health centers. 

• Family services. Services include family needs assessments; family case management, 
linking to community resources; parenting groups; parent/guardian education. 

• Community education and development. Services include community needs assessment; 
out-of-school-time activities; adult English literacy classes; other adult education; 
community-wide events. 

This report, third in a series of studies addressing the implementation and outcomes of the Linkages 
to Learning initiative, examined whether students at LTL sites who received mental health services 
and/or had family members receive case management services demonstrated improvement in 
levels of well-being and school engagement. The study used a pre-post design: to assess change 
in status, relevant measures (student and family indicators) were analyzed at the time of intake for 
mental health or case management services or in the year prior to LTL participation, and after 
participation in these LTL services. Student outcomes were assessed with the use of nationally 
validated measures and MCPS student data; family outcomes were assessed with a matrix widely 
used by community action agencies to assess client needs and progress toward self-sufficiency, as 
well as locally developed measures. 

The specific questions addressed in this report were: 

• To what extent did students who received mental health services and/or whose 
families received case management services at an LTL site show improvement on 
measures of well-being and school engagement? 

o How did students rate their well-being at the start and end of participation in LTL 
child/family therapy services? 

o How did teachers rate the behavior of students at the start and end of participation 
in LTL child/family therapy services? 

o Was there a change in school attendance after participation in LTL mental health 
or family case management services? 

o Was there a change in Work Habits grades for elementary students during the year 
LTL mental health and/or family case management services were received? 
o Among middle school students who were suspended in 2013-2014 and received 
LTL mental health and/or family case management services, what was the rate of 
repeat suspension in 2014-2015? 

• After participating in Linkages to Learning case management services, to what extent 
did families show improvement in well-being and support of their child’s education, 
and increased community participation? 
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o Did parents/guardians show an increased capacity to meet basic needs? 
o Did families show increased support of students’ education at home and at school 
and increased community participation? 


Summary of Findings 

The key findings are summarized below. 

1 . Students who received mental health services at an LTL site showed improvement on some 
measures of well-being and school engagement. On self-ratings of self-concept at referral to 
LTL and when exiting, elementary students improved on five of six scales plus an overall 
measure, and middle school students improved on two of five scales plus an overall measure. 

2. Teacher ratings of classroom behavior were not significantly different for students before and 
after they received mental health services through LTL. 

3. Attendance (percentage of days attended) for elementary students who participated in 
recreation groups through LTL was statistically significantly higher during the year of LTL 
participation compared to the previous year. For middle school students whose families 
received case management services, attendance rate was significantly lower during the LTL 
service year, but among middle school students who participated in recreation activities 
through LTL, the number of unexcused absences was significantly fewer compared with the 
previous year. An examination of suspension rates for students who were suspended during 
the year they received LTL services showed that nearly three quarters of them (72%) were not 
suspended during the following school year. 

4. Families who received case management services showed improvement on most areas of self- 
sufficiency. Ratings on health, nutrition, family development, income management, adult 
education, and community participation were statistically significantly higher after receiving 
LTL family case management services than at referral to LTL. 

5. In an end-of-year LTL survey, large percentages of parents/guardians (more than 90%) agreed 
that LTL had helped them support their students’ education and that LTL had helped their 
student and their family feel a part of school and become more engaged. 


Recommendations 

1 . Utilize technology to strengthen the process for therapists and case managers to receive daily 
feedback about the attendance of students with whom they are working. Some evidence of 
improvements in attendance was observed for students receiving services in LTL sites, 
particularly for elementary students. Since school attendance is a key ingredient for academic 
success, closer, more real-time monitoring may strengthen LTL’s impact on attendance. 
Currently, LTL staff have access via parent/guardian consent to the MCPS student database; 
however, the database is not updated in real time. As such, outside of bi-weekly team meetings 
and direct contact with students and family members being served, LTL staff rely on 
notification from teachers, counselors, or other school-based staff. Workgroups are underway 
between MCDHHS and MCPS to develop a data-sharing process that will allow relevant 
information to be accessed more efficiently. 
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2. Explore ways to increase systematic collaboration between LTL sendee providers and 
teachers to better understand specific issues and concerns of students who are not 
demonstrating skills associated with expected work habits. About one half of the students 
receiving mental health services and/or whose families were receiving case management at 
LTL sites were demonstrating the skills related to two report card work habits — Task 
Completion and Rules and Procedures — in marking period 4. 

3. Explore additional ways to support families in areas that appear most challenging to impact. 
Many areas of family self-sufficiency showed statistically significant improvement after 
family members received LTL case management services. Some areas — such as employment 
and income management — did show improvement during LTL participation, but remain 
challenging for many families. Difficult problems such as these require additional resources 
and innovative approaches. LTL has been looking into new partnerships with Montgomery 
College to expand adult education opportunities offered on-site at LTL schools in hopes of 
improving opportunities for employment for parents/guardians. LTL should explore additional 
systemic partnerships to support outcomes in these areas. 

4. Institute more regular and consistent administration of LTL program assessments. A relatively 
small number of students who exited LTL mental health services during the 2013-2014 school 
year completed measures of well-being at referral and when exiting LTL services; likewise, 
few students had behavior ratings completed by teachers. Ratings of family self-sufficiency 
also were completed for a relatively small number of families who received case management 
services at LTL sites, although larger numbers have been collected in subsequent years. (The 
year reviewed in this study was only the second year for administering the family self- 
sufficiency scale. In order to obtain consistent baseline measures, the scale was administered 
only with families who were new to the program at intake, not with families who had been 
previously receiving LTL case management services.) Obtaining LTL measures for all 
students and families who receive LTL services is needed so that a more reliable summary of 
the status of clients’ well-being can be reported, more regular feedback can be provided to 
program staff, and additional infonnation is available for monitoring and adjusting services. 
LTL is currently exploring the use of new electronic assessment measures in order to achieve 
this improvement. 

5. Increase means for feedback from students participating in LTL sendees. At the time of data 
collection for this study, needs surveys were administered to parents/guardians in schools with 
an LTL site, satisfaction surveys were administered each year to parents/guardians who 
received (or their children received) any LTL services, data scales were collected from students 
who received behavioral health services, family self-sufficiency scales were collected with 
families receiving case management services, and surveys for specific events and programs 
were collected. However, no consistent satisfaction surveys of students receiving LTL services 
were conducted across the initiative. It is recommended that two versions of surveys be 
administered at all LTL sites: (1) to a sample of all students in the school to identify ways to 
make services more accessible and to target needs, interests, and ideas for groups and activities 
(to be coordinated with MCPS protocols for student survey administration); and (2) to students 
who have received any LTL service to ascertain what aspects of LTL have been most helpful 
and what additional services are needed. It should be noted that student needs and satisfaction 
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surveys are now in place at LTL middle school sites, with plans underway to expand to higher 
elementary school grades in the future. 
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Evaluation of Linkages to Learning: Report on the Status of Students and 
Families Who Received Mental Health and Family Case Management 
Services at Linkages to Learning Sites, 2013-2014 

Julie Wade, M.S. and Nyambura Maina, Ph.D. 

The Office of Shared Accountability (OSA) conducted an evaluation of the Linkages to Learning 
initiative in Montgomery County Public Schools (MCPS). This report, third in a series of studies 
addressing the implementation and outcomes of the Linkages to Learning initiative, examines 
whether students and family members who received mental health and family case management 
services at LTL sites demonstrated improvement in levels of well-being and school engagement. 

Linkages to Learning (LTL) is a collaborative initiative among the Montgomery County 
Department of Health and Human Services (MCDHHS), (MCPS), and local non-profit agencies. 
The goal of the program is to address the social, economic, health, and emotional issues that 
interfere with the academic success of a child (MCDHHS, 2015). LTL’s collaborative model 
brings together providers within Montgomery County to deliver services that are integrated, 
accessible, needs-driven, and community-based (Weast, 2005). For more than 20 years, LTL has 
provided accessible services to at-risk children and their families in MCPS, with the goals of 
improving student well-being and success at school, home, and in the community (MCDHHS, 
2015). Parents/guardians also serve as leaders in LTL, working as partners with program staff to 
develop strengths-based, culturally appropriate solutions to the challenges confronting their 
children, schools, and communities. Prevention and early intervention services include health and 
behavioral health services, social services, and community education and development (including 
after-school and family programming, and adult education classes). Today the initiative includes 
29 sites in the Clarksburg, Gaithersburg, Wheaton, and Rockville clusters, the Northeast 
Consortium, and the Downcounty Consortium. Twenty-three of the sites are in elementary schools 
and six are in middle schools. Schools with an LTL site are listed in Appendix A, Figure A-l. 


Background 


Overview of Services Provided at Linkages to Learning Sites 

To achieve its goals, LTL focuses its services on addressing three broad areas of need: 

• Student well-being. Services include assessment for social-emotional, behavioral 
concerns; consultation with teachers; child/family/group therapy; psychosocial skill 
development groups; primary care and treatment at LTL school-based health centers. 

• Family services. Services include family needs assessment; family case management; 
linking to community resources; parenting groups; parent/guardian education. 

• Community education and development. Services include community needs 
assessment; out-of-school-time activities; adult English literacy classes; other adult 
education; community- wide events. 
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The three service areas that LTL addresses — student well-being, family services, and community 
education and development — became the framework for a logic model that was developed in 2013 
in a collaboration of MCPS evaluators, the LTL Advisory Group Program & Planning Committee, 
and consultation and assistance from Casey Family Programs. The logic model is depicted in 
Appendix A. Within each of the service areas, Linkages to Learning provides a range of activities 
and services for students and their families. The implementation of the activities detailed in each 
service area is expected to contribute to the realization of a series of outputs and short- and long- 
term outcomes, as depicted in the logic model. Since this updated logic model was constructed 
after the initiative had been in place for many years, the model was retrofitted to incorporate the 
existing services and activities, and readily available data related to the model’s outcomes were 
utilized for this evaluation. A description of services and activities provided at LTL sites is shown 
in Appendix B. 

Staffing at LTL sites. LTL is staffed by a multidisciplinary team at each of the school sites. The 
full staffing model 1 includes one full-time community school coordinator, one full-time family 
case manager, and one full-time child/family therapist to work at each school in conjunction with 
school staff. The full staffing model at LTL School-Based Health Centers (SBHC) includes a part- 
time community services aide, though not all SBHC sites have this. Nine of the elementary schools 
with LTL have SBHCs (see Appendix C), where pediatric nurse practitioners work with a full- 
time school nurse and school health aide to provide primary health care services, including sick 
care, immunizations, and physical examinations. 

Family case management services. The LTL family case manager works with families to help 
them become more able to provide for their basic needs. The family case manager conducts family 
needs assessments, links families to needed resources and benefits, helps families navigate the 
school system and become more engaged with their children’s education, and helps families access 
support groups, parenting groups, adult English literacy classes, and adult education. 

Child and family mental health services. The LTL child/family therapist works directly with 
students who are referred for individual/family therapy or participation in group therapeutic 
activities, such as self-esteem groups, psychosocial skills groups, or other groups based on student 
needs. Students are referred to the therapist either directly by the school counselor or by the 
collaborative problem solving team at each school site. 

Purpose and Scope of the Study 

This report is the third in a series of reports addressing the implementation and outcomes of the 
LTL initiative. The goal of this section of the evaluation was to examine potential progress in the 
well-being and engagement of students and families who received mental health services and/or 
family case management services at LTL sites. Separate reports addressed evaluation questions 
focused on the implementation of the initiative and school-level differences. 


1 Not all schools have the staff designated in this model. Relatively flat funding at older sites and budget cuts 
implemented during the recession have left many schools with some part-time staff. 
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The specific questions addressed in this report were: 

• To what extent did students who received mental health services and/or whose 
families received case management services at an LTL site show improvement on 
measures of well-being and school engagement? 

o How did students rate their well-being at the start and end of participation in LTL 
child/family therapy services? 

o How did teachers rate the behavior of students at the start and end of participation 
in LTL child/family therapy services? 

o Was there a change in school attendance after participation in LTL mental health 
or family case management services? 

o Was there a change in Work Habits grades for elementary students during the year 
LTL mental health and/or family case management services were received? 
o Among middle school students who were suspended in 2013-2014 and received 
LTL mental health and/or family case management services, what was the rate of 
repeat suspension in 2014-2015? 

• After participating in Linkages to Learning case management services, to what extent 
did families show improvement in well-being and support of their students’ education 
and increased community participation? 

o Did parents/guardians show an increased capacity to meet basic needs? 
o Did families show increased support of students’ education at home and at school 
and increased community participation? 

Review of Selected Literature on School-Linked Services 

Children living with social and economic challenges, such as poverty, physical and mental health 
problems, and unstable living conditions, are less likely to succeed in school and thrive 
intellectually, physically, and emotionally (Anderson-Butcher & Ashton, 2004; Barton & Coley, 
2009; Morsy & Rothstein, 2015; Sandstrom & Huerta, 2013). Several approaches have emerged 
that aim to address the interconnected needs of students’ learning and their physical and emotional 
health. A guiding premise of these efforts is the recognition that “the development of a child is 
affected by a range of proximal and distal influences and contexts, and that multiple influences 
can be leveraged to enhance the development of a child or adolescent over time” (Child Trends, 
2014, p. 75). 

Evaluation of Linkages to Learning in MCPS. 

In 1999, researchers from the University of Maryland conducted an evaluation of LTL at one 
MCPS elementary school (Fox et ah, 1999). The study compared academic (mathematics, reading, 
and writing) and nonacademic (positive and negative behaviors, emotional distress, 
parents’/guardians’ mental health) outcomes for children and parents/guardians at an LTL school 
and a comparison school without LTL. Findings revealed a decrease in parent/guardian-reported 
negative behaviors for students in the LTL school, while the negative behaviors of students in the 
non-LTL school remained the same. Examination of academic outcomes suggested some positive 
effects of educational services provided by LTL. Students at the school with LTL and the 
comparison school without LTL had significantly higher math achievement scores at the end of 
the study, but students who received educational services through LTL improved significantly 
more than those at the same school who did not receive these services. 
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Evaluation of school-linked services. 

Several recent reports provide findings for studies examining the impact of school-linked services 
on students’ school engagement and academic perfonnance. Castrechini and London (2012) 
studied community schools in Redwood City, California, where Family Resource Centers at the 
public elementary and middle schools provide integrated services to address students’ and 
families’ physical, social, emotional, and learning needs. To examine students’ involvement 
across a range of programs and supports, the researchers classified the nearly 250 programs and 
activities into three main “strategy areas”: family engagement, extended learning, and support. 
Analyses included examination of participation by demographic characteristics and number of 
strategy areas accessed. The study found a relationship between family engagement and English 
language proficiency, particularly in early grades, but no direct links between community school 
program participation and standardized mathematics and English language performance. Students 
who participated in community school programs, specifically in family engagement and extended 
learning, reported feeling more supported than students with no participation; and the researchers 
found that students’ feelings of support had a significant effect on their motivation and confidence, 
suggesting a possible mediating effect. The researchers conclude that: 

“Evidence in this study and others shows that when students feel connected to their schools, there 
are important positive implications for their academic motivation and confidence, both of which 
have strong li nk s to academic perfonnance.... Findings showing a strong link between positive 
student attitudes and supplemental community school programs suggest that community schools, 
through the multiple opportunities they provide for positive interactions between students and their 
schools, could be a promising strategy for fostering positive attitudes that promote achievement” 
(Castrechini & London, 2012, p. 25). 

An evaluation of Communities in Schools (CIS) examined the impact of CIS at the organizational 
level, the school level, and the student level using multiple measures and methods (ICF, 2010). 
Communities in Schools is a national organization with about 200 local affiliates serving more 
than 1 million students in more than 3,000 schools. The organization places a dedicated staff 
member inside partner schools who works with school staff to engage community partners and 
volunteers to effectively and efficiently address the academic and social service needs of students. 
This five-year national evaluation included a range of studies: an analysis of the level of 
implementation; a quasi-experimental study to analyze school-level outcomes; case studies; 
comparisons with peer organizations; and randomized controlled trials. The evaluation found 
positive effects (effect size greater than .25) for CIS schools on dropout and graduation rates, 
attendance in elementary schools, middle school math and reading perfonnance, and student 
perception of personal responsibility. The strongest effects were seen in schools that implemented 
the model with the highest fidelity (ICF, 2010). 

A review of the literature in school-community partnerships by Valli, Stefanski, and Jacobson 
(2014) examined student and family outcomes as well as organizational outcomes. Their study 
found that the most frequently studied outcome, and the one with the most consistently positive 
findings, was academic achievement, particularly in mathematics. Evidence for improvement in 
literacy, attendance, classroom behaviors, and attitude toward school also were seen across the 
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studies reviewed for their report. Valli, Stefanski, and Jacobson (2014) also described factors that 
were identified as important intervening variables for academic achievement, including fidelity of 
program implementation and continuity of the services over time, as well as trust in the schools 
and family engagement. 

Predictors of educational achievement. 

Much of the support provided by school-linked service approaches is not linked directly to 
academic outcomes. Instead, the programs target the needs of students and their families that 
would interfere with academic success. The Coalition for Community Schools’ logic model 
(Coalition for Community Schools, 2009), for example, posits that providing services to support 
the social, physical, cognitive, and economic needs of students and their families will help ensure 
that the conditions necessary for learning are strengthened and students have greater chance for 
success. 

A review of the literature on factors related to educational achievement conducted by Child Trends 
(Princiotta et ah, 2014) identified the strongest predictors of educational success. The authors 
focused on factors that were malleable, so as to be amenable to programmatic or policy changes. 
The review identified factors concentrated in three domains: individual student factors, 
parent/guardian and family factors, and school factors. Those with the largest observed effects 
were: 

Individual student factors 

• Behavior in and out of class 

• Attendance/engagement 

• Academic self-concept/self-efficacy 

• Social and emotional competence 

• Health and well-being 
Parent and family factors 

• Parental expectations 

• Parenting style 

• Parenting behaviors 
School factors 

• Effectiveness of teachers 

• School climate, including student-teacher relationships, academic press 

• Average school achievement and attendance 

• School-level poverty 

This study examined some of these factors for students and parents/guardians receiving services 
at LTL sites. 


Methodology 


Evaluation Design 

This study was designed to examine the extent to which students and family members who 
received mental health and/or case management services at LTL sites were making progress in 
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measures of well-being and engagement. The evaluation questions were addressed with a pre-post 
design: relevant measures (student and family indicators) were analyzed at baseline (at intake into 
services, or in the year prior to LTL participation) and after LTL participation (depending on 
measure, either approximately six months after intake or at end of treatment for the 2013-2014 
school year) to assess potential change. When the number of students was sufficient, some data 
were analyzed and reported within school level (elementary and middle school) and by type of 
LTL services received. 

Study Schools 

Twenty-six of the current twenty-nine LTL schools had LTL in 2013-2014 and were included in 
the study (20 elementary schools and six middle schools). Three sites that opened in 2014 and 
2015 (where LTL had been operating for less than one year) were not included in the study. 

Study Sample 

Information included in this study is specific to the students and families who received family case 
management or mental health services at an LTL site during the 2013-2014 school year. A total 
of 2,120 students and 1,025 family members had records of family case management and/or mental 
health services from LTL during the 2013-2014 school year. 

Outcome Measures 

Student and family outcomes were assessed with the use of nationally validated instruments as 
well as locally developed measures, and with MCPS student data. Some outcome measures, such 
as attendance, were examined for all students in the study, but some outcome measures were 
administered only to subsets of the students and family members with LTL service records. For 
example, the measure of student self-concept was administered only to children who received child 
or family therapy (not to those who only received group mental health services), and the instrument 
was not appropriate for children younger than second grade. The measure of family self- 
sufficiency was used only with families who were new to LTL services, not with families who 
were continuing services. 


Student self-concept. The Piers-Harris Children’s Self-Concept Scale, Second Edition (Piers- 
Harris 2; Piers, Harris, & Herzberg, 2002) was administered to students receiving mental health 
services at LTL sites to assess their self-perceived well-being at the time of intake and at planned 
discharge. The Piers-Harris 2 is a nationally nonned measure of children’s (age 7 to 18 years) 
psychological health. The scale assesses student’s behavioral adjustment, student’s self- 
perception of intellectual and academic abilities, student’s self-esteem, student’s report of anxiety 
and dysphoric mood, and student’s happiness and satisfaction with life. The instrument consists 
of 60 items to which the child responds “yes” or “no.” Analyses of the psychometric properties of 
the scale yielded acceptable reliability and validity. A Chronbach’s alpha of .91 was reported for 
the total scale, and values ranging from .74 to .81 were reported for the subscales. Convergent 
validity of the original Piers Harris scale was examined with correlational studies of related 
constructs; in general the results demonstrated that a positive self-concept is inversely related to 
measures that target psychological problems. LTL staff administered the form to students referred 
for child/family therapy at the time of intake, and whenever possible, at the time of discharge. 
Since some terminations are unplanned, not all post-LTL administrations of the instrument can be 
administered. 
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Student classroom behavior. Teachers rated the classroom behavior of students receiving 
child/family therapy services using the Sutter-Eyberg Student Behavior Inventory, Revised 
(SESBI-R; Eyberg & Pincus, 1999). The SESBI-R is used to assess conduct problems in youth 
aged 2 to 16. The SESBI-R has shown high internal consistency (Chronbach’s alpha was .98 and 
.96 on two scales). The instrument consists of 38 items rated on a seven-point scale from “Never” 
to “Always,” and a yes/no indication of whether the behavior is a problem for the rater (the 
teacher). LTL staff worked with classroom teachers to arrange for completion of the rating scale 
at the time of referral and when the student exited child/family therapy services. Some SESBI-R 
ratings were completed by different teachers at intake and discharge, so changes in student 
behavior may be confounded by variability in raters. 

Work Habits grades. For elementary students, MCPS report card grades for Work Habits (Task 
Completion and Rules and Procedures) were used to examine student behavior from Marking 
Period 1 to Marking Period 4 of the year that LTL services were received (2013-2014). Grades on 
Work Habits reflect the effort and behaviors that affect learning Work Habits grades for the same 
year were used so that the grades at the two marking periods would be assigned by the same 
teacher; only grades for students who were receiving LTL services during both Marking Period 1 
and Marking Period 4 were included in the analysis. 

Student school attendance. Student attendance data (percentage of days attended during school 
year; number of unexcused absences during school year) were used to measure attendance during 
the year prior to LTL services (2012-2013) and during the year of LTL services (2013-2014). 
Data for students in Grades 1 through 8 during 2013-2014 were included in the analysis. 

Suspensions. Among middle school students who were suspended in 2013-2014 and received 
LTL services, the rate of repeat suspension in 2014-2015 was examined. 

Family self-sufficiency rating. The Massachusetts Family Self-Sufficiency Scales and Ladders 
Assessment Form (Massachusetts Department of Housing and Community Development, 1999) 
was used with families receiving LTL case management services to assess their status in several 
key areas. The form was administered by the LTL case worker during the intake process and 
approximately every six months until the family exited services. Families were rated on a live- 
level scale from “in crisis” to “thriving” in areas of employment, child’s education, adult 
education, health, childcare, youth and development, housing, income management, 
transportation, community participation, and nutrition. 

Parent’s/guardian’s perceptions of student’s engagement in school. The LTL parent survey 
assessed parent’s/guardian’s perceptions of student’s engagement in school and attitude toward 
school, as well as parent’s/guardian’s engagement in student’s education. Parents/guardians who 
received LTL services during 2013-2014 were asked to complete a survey; surveys were collected 
by LTL staff from April through June, 2014. Surveys were available in English and Spanish and 
were completed with paper and pencil; respondents were not identified by name. The estimated 
response rate was 23% (based on number of parents/guardians receiving services during 2013— 
2014). 
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Data Sources 

LTL program records provided data for students and families who received LTL case management 
or mental health services during the 2013-2014 school year. Staff from LTL provided secure files 
containing LTL service records for analysis by OSA. 

LTL program assessment data was used for analyses of student and family outcomes (student 
self-concept; teacher-rated student behavior; family self-sufficiency). Staff from LTL provided 
secure files containing pre- and post-LTL assessment data for analysis by OSA. 

MCPS student records provided demographic data for the students and were used to examine 
student attendance, numbers of unexcused absences, behavioral measures/leaming behaviors on 
elementary report cards, and suspension data for middle school students. 

Analysis of Data 

Paired t-tests were used with means (continuous data), and McNemar’s tests 2 were used with 
percentages (dichotomous data) to address the first set of evaluation questions — students’ 
measures of well-being and engagement. Descriptive statistics (percentages) and McNemar’s tests 
were used to report findings related to the second set of evaluation questions — parent/guardian and 
family well-being and support of child’s education and engagement. Specific analyses are 
described below for each evaluation question. 

Evaluation Question: To what extent did students who received mental health services and/or 
whose families received case management services at an LTL site show improvement on 
measures of well-being and school engagement? 

• Two sets of paired t-tests were conducted with student measures: (1) analysis of students’ self- 
ratings of their well-being and (2) analysis of teachers’ ratings of student behavior. Each of 
these measures were analyzed at two time periods: at referral to LTL and after receiving LTL 
services during the 2013-2014 school year. Ratings on the two instruments (dependent 
variables) were analyzed over the two time periods (independent variable). 

• Attendance measures (mean percentage of days attended and mean number of unexcused 
absences during the school year) were analyzed with paired t-tests; the independent variable 
was time (previous school year and year of LTL service), and dependent variables were mean 
percentage of days attended and mean number of unexcused absences. Analyses were 
conducted separately for elementary and middle school students. 

• Learning Skills (Work Habits and Rules and Procedures) were examined for elementary 
students at two points in time: Marking Period 1 and Marking Period 4, and the percentages 
of students demonstrating the skill at each time period were compared using McNemar’s test. 
Only students who were receiving LTL services during the two marking periods were included 
in the analysis. 

• The rate of repeat suspensions in 2014-2015 was examined descriptively for middle school 
students who were suspended in 2013-2014 and received LTL services. 


2 McNemar's test is a statistical procedure used on paired nominal data to test whether the two related samples have 
the same distribution. 
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Evaluation Question: After participating in Linkages to Learning services, to what extent 
did families show improvement in well-being and support of their student’s education and 
increased community participation? 

• Findings related to family well-being were reported descriptively and pre- and post-LTL 
ratings were compared with McNemar’s test. The percentage of families rated at each of five 
levels of self-sufficiency at referral and after LTL services were reported. Change in ratings 
over the course of LTL involvement was analyzed by comparing the percentages of families 
rated Safe, Stable or Thriving at referral and after receiving LTL services, using McNemar’s 
test. 

• Percentages of parents/guardians agreeing with post-LTL survey items pertaining to a child’s 
engagement in school were reported descriptively. 

In addition to tests of statistical significance, effect sizes (Cohen’s d statistic, see Appendix D) 
were calculated to estimate the magnitude of the change and whether the difference is meaningful 
in an educational setting (American Psychological Association, 2010; Cohen, 1988). Cohen 
(1988) suggested that d= 0.2 be considered a “small” effect size, 0.5 represents a “medium” effect 
size, and 0.8 a “large” effect size. In analyses of “paired” measures, such as pre- and post-service 
measures that compare the scores for the same subjects over time, the effect size must consider the 
correlation between the measurements; this correction is accomplished with a d z statistic (Lakens, 
2013). 
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Summary of Methodology 

Table 1 outlines the outcome measures, instruments, data collection, and analytic procedures used 
in the study. 


Table 1 


Summary of Measures and Data Collection Procedures for 
Students and Family Members Receiving LTL Services, 2013-2014 


Outcome Measure 

Instrument/ 

Data Source 

Number and 
schedule of 

measures 

Number of 
students/family 
members with 
data 3 

Analysis 

To what extent do sti 
an LTL site show im l 

idents who have received family case management and/or mental health services at 
provement on measures of well-being and school engagement? 

Student self- 
concept 

Piers-Flarris Children’s 
Self-Concept Scale; 
administered by LTL 

2 administrations: 
at referral, after 
services 

58 students 

Pre-, post-LTL 
mean 

differences 

Student classroom 
behavior 

Sutter-Eyberg Student 
Behavior Inventory; 
administered by LTL 

2 administrations: 
at referral, after 
services 

66 students 

Pre-, post-LTL 
mean 

differences 

Student classroom 
work habits 

Elementary report card 
grades on Task 
Completion, Rules, and 
Procedures; MCPS 
records 

2 measurements: 
Marking periods 
(MP) 1 and 4 

682 students in 
Grades 1-5 who 
were receiving LTL 
services during 

MP1 and MP4 

% students 
demonstrating 
skill, MP 1, MP 

4 

School attendance 

% days attended; 
number of unexcused 
absences; MCPS 
records 

2 measurements: 

2012- 2013 and 

2013- 2014 school 
years 

Elementary 
students: 1,090; 
Middle school 
students: 572 

Pre-, post-LTL 
mean 

differences 

Suspensions 

Number and % students 
with suspension; 
number and % students 
repeating suspension 
after LTL year; MCPS 
records 

Data reported for 

2 school years: 
2013-2014; 2014- 
2015 

61 MS students 
who were 
suspended in 2013- 
2014 and received 
family case 
management and/or 
MH services 

Descriptive, 

percentages 

After participating in Linkages to Learning services, to what extent did families show improvement 
in well-being and support of their student’s education, and increased community participation? 

Family self- 
sufficiency 

The Massachusetts 

Family Self-Sufficiency 
Scales and Ladders 
Assessment Form; 
administered by LTL 

2 administrations: 
at referral, after 
services 

122 

parents/guardians 

Pre-, post-LTL 
% families 
rated safe or 
better 

Parent engagement 
and perceptions of 
student’s school 
engagement 

LTL Parent Survey; 
administered by LTL 

1 administration: 
at the end of the 
school year 

299 

parents/guardians 

Descriptive, 
percentages or 
mean ratings 


a Data for students and family members were included only when both pre- and post-LTL assessments or records were 
available. 
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Strengths and Limitations of the Methodology 

Strengths. To ensure that the evaluation addressed the issues of most importance and interest to 
the administrators of the initiative, the evaluation plan was developed in collaboration with the 
Linkages leadership team, comprising administrators from HHS, MCPS, and the partner agencies. 
In addition, the stakeholder surveys were developed with input collected from LTL community 
school coordinators, who coordinate the Linkages services and activities at the school sites. 

The evaluation used published measures and one locally developed survey. The Massachusetts 
Family Self-Sufficiency Scales and Ladders Assessment Form is a widely-used instrument for 
assessing levels of well-being in key life areas. The Piers-Harris Children’s Self-Concept Scale is 
one of the most widely used measures of psychological health in children and adolescents, and the 
Sutter-Eyberg Student Behavior Inventory has been shown to be a reliable measure of student 
problem behaviors (Floyd et al., 2004). In addition, a parent/guardian survey was developed to 
obtain parents ’/guardians’ perceptions of their child’s school engagement and their own 
involvement in their child’s education. The survey was developed by evaluators in collaboration 
with LTL administrators. 

In a program like LTL, with a wide array of services and activities, students and their families may 
participate in different services and combinations of services, and for different amounts of time. 
To ensure that students and families had a minimum of four months in LTL, analyses did not 
include those whose initial intake for LTL was April or later or for whom the number of months 
between intake and discharge was fewer than four. In addition, when the numbers of participants 
were sufficient, separate analyses were conducted for groups of students who received different 
types of services. 

Limitations. The use of self-ratings on the Piers-Harris, teacher ratings on the Sutter-Eyberg, and 
case manager ratings on the Massachusetts Scales introduces the potential for bias, since all raters 
were aware that the student or family member being rated was referred to LTL, and subsequently 
received services from LTL. 

Different amounts of services, over different amounts of time, were received by students and 
family members in LTL. In addition, the time between pre-LTL and post-LTL assessments varied. 
All students and family members in the study received services during the 2013-2014 school year, 
but they may have been referred at the start of the school year or later in the year (students and 
family members referred in April or later were not included). Some may have received services 
in a previous school year as well. Therefore, although the study included measures at two time 
periods, different amounts of services may have been received between the pre-LTL and post-LTL 
measures. 

No appropriate comparison groups were available for the analyses in this report. Since the group 
of interest in this study comprised students or family members who received LTL services, the 
“defining characteristic” — the need for services — is not a characteristic that can be matched in a 
comparison group in this study. Therefore, the pre-post design was used, but it must be understood 
that observing changes in the study (LTL) group without an examination of change in an untreated 
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group leaves open the possibility that the change was due to other factors (e.g., maturation, time 
of year) and that it may have occurred without LTL. 

In analyses of student self-concept, student classroom behavior, and family self-sufficiency, the 
number of students and family members who had pre- and post-LTL assessments was small 
compared with the number of those who received LTL services because the instruments or the 
scheduling of them were not appropriate for all who received services. As a result, the analyses 
of student or family progress using these measures were conducted using small numbers of ratings 
and did not allow separate analyses by type of service received. 

Results 

Evaluation question: To what extent do students who have received family case management 
or mental health services at an LTL site show improvement on measures of well-being and 
school engagement? 

The demographic characteristics of students who received LTL family case management or mental 
health services during the 2013-2014 school year are shown in Table 2. 


Table 2 

Demographic Characteristics of Elementary and Middle School Students 
who Received Case Management or Mental Health Services at LTL sites, 2013-2014 


Characteristics 

N = 2,125 


Grade Level 

n 

% 

Kindergarten, 1 st , 2 nd 

560 

26.4 

3 rd , 4 th 

509 

23.9 

5 th 

285 

13.4 

6 th 

257 

12.1 

yth 

237 

11.2 

8 th 

192 

9.0 

9* a 

85 

4.0 

Race/Ethnicity 

Black or African American 

370 

17.4 

Asian American 

38 

1.8 

Hispanic/Latino 

1,556 

73.2 

White 

107 

5.0 

Two or More Races 

49 

2.3 

Gender 

Female 

1,026 

48.3 

Male 

1,099 

51.7 

Service Provided 

ESOL (current) 

814 

38.3 

Spec. Educ. (current) 

484 

22.8 

FARMS (current or prior) 

1,994 

93.9 


Note. American Indian students were not included because n < 10. 

a Students in 9 th grade who received LTL family case management or mental health services were those 
who were referred previously in middle school and were still receiving services in 2013-2014. 
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The demographic characteristics of the students who received mental health or family case 
management services through LTL were similar to that of the school populations in the LTL sites, 
particularly compared with the segment of the school population receiving FARMS, which is the 
target group for services provided by LTL. A description of the characteristics of LTL schools 
and the students receiving services is included in a previous report (Wade & Maina, November 
2015). 

How do students rate their well-being at the start of and after LTL participation? 

For a subset of students receiving mental health or family case management services at an LTL 
site ( N = 58), self-ratings of their well-being were available at referral to LTL and after LTL 
participation. Table 3 shows the characteristics of the students with pre- and post-LTL ratings on 
the Piers-Harris 2. 


Table 3 

Demographic Characteristics of Elementary and Middle School Students With 
Pre-LTL and Post-LTL Piers-FIarris 2 Scores, 2013-2014 


Characteristics 

OO 

II 


School Level 

n 

% 

Elementary 

30 

51.7 

Middle School 

28 

48.3 

Race/Ethnicity 

Black or African American 

14 

24.1 

Asian American 

<5 

— 

Hispanic 

36 

62.1 

White 

<5 

— 

Two or More Races 

<5 

— 

Gender 

Female 

27 

46.6 

Male 

31 

53.4 

Service Provided 

ESOL (current) 

15 

25.9 

Spec. Educ. (current) 

18 

31.0 

FARMS (current or prior) 

57 

98.3 


The characteristics of the students with Piers-Harris 2 ratings were similar, for the most part, to 
the characteristics of all students who received LTL services (see Table 2). The LTL services 
received in 2013-2014 by the students with Piers-Harris 2 data are shown in Table 4. It should be 
remembered that students may have received more than one type of service, and each service may 
have been received multiple times. Most of the students with Piers-Harris 2 data received family 
case management services and individual psychotherapy; about two thirds received family therapy. 
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Table 4 

Services Received by Students With Pre-LTL and Post-LTL Piers-Harris 2 Scores (N = 58) 


Type of mental health or case 
management service 

Number of students 
with service 

% of students 
with service 

Median times service 
received /participated 3 

Family case management 

54 

93.1 

7 

Family therapy 

38 

65.5 

2 

Individual therapy 

56 

96.6 

20 

Group therapy 

23 

39.7 

6 


a Number of service encounters includes services provided for students and family members; services were received during 


2013-2014. 


Responses of elementary students who completed the Piers-Harris 2 at referral to LTL and after 
LTL participation are shown in Table 5. Self-rated well-being scores were significantly higher (p 
< .05) after LTL participation on five of the six scales and on the total score representing “General 
self-concept” {p < .01). Effect sizes were in the small to medium range, using Cohen’s criteria, 
indicating that the increases in Piers-Harris 2 scores were of practical significance. “Freedom from 
anxiety” had the largest effect size of the six scales for elementary students. 


Table 5 

Self-Ratings of Well-Being on Piers-Harris 2 by Elementary School Students With 
LTL Services in 2013-2014 at Referral (Pre-Service) and After Participation (Post-Service) 

Pre-Service Post-Service 

T score T score 

Paired 


Scales 

N 

Mean 

Mean 

Mean 

difference 

Std. 

error 

t 

P 

Effect 
size ( d : ) 

Total score/ 

General self-concept 

30 

49.73 

56.43 

6.70** 

2.16 

3.11 

.004 

.57 

Behavioral adjustment/ 

Self-perception of positive 
and negative behaviors 

30 

50.80 

54.90 

4.10* 

1.84 

2.23 

.034 

.41 

Intellectual status/ 

Self-assessment of intellectual 
and academic abilities 

30 

51.70 

56.33 

4.63* 

2.00 

2.31 

.028 

.42 

Physical appearance & attributes/ Self- 
appraisal of appearance, leadership, 
ability to express ideas 

30 

49.87 

53.80 

3.93* 

1.63 

2.41 

.023 

.44 

Freedom from anxiety/ 

Positive emotional states, not 
bothered by sadness, worry, 
unpleasant mood 

30 

49.57 

55.03 

5.47* 

2.06 

2.65 

.013 

.48 

Popularity/ 

Self-evaluation of social 
functioning, peer relationships 

30 

46.33 

50.87 

4.53* 

1.90 

2.39 

.024 

.44 

Happiness and satisfaction / 

Feelings of happiness and 
satisfaction with life 

30 

53.93 

55.80 

1.87 

1.43 

1.31 

.200 

.24 


Note. T scores are standardized scores with a mean of 50 and a standard deviation of 10. On all Piers-Harris 2 scales except the total 
score, T scores from 40 to 55 are considered to be within the average range. On total score, T scores between 40 and 59 comprise 
the average range. On all scales and total score, higher scores represent higher ratings of well-being. 


* p < .05; ** p < .01 
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Among the middle school students who completed Piers-Harris 2 at referral to LTL and after 
receiving services at LTL sites (Table 6), their wellbeing scores were significantly higher after 
receiving services on two of the six scales — “Freedom from anxiety” and “Happiness and 
satisfaction” — and on the total score representing “General self-concept” (p < .05). Effect sizes 
for differences on these three scales were between small and medium (.40 to .56), indicating that 
the differences were large enough to be of practical significance. 


Table 6 

Self-Ratings of Well-Being on the Piers-Harris 2 by Middle School Students 
With LTL Services in 2013-2014 at Referral (Pre-Service) and After Participation (Post- Service) 

Pre-Service Post-Service 

T Score T Score 

Paired Effect 



N 

Mean 

Mean 

Mean 

difference 

Std. 

error 

t 

P 

size 

(d : ) 

Total score/ 

General self-concept* 

28 

44.50 

48.50 

4.00* 

1.87 

2.14 

.042 

.40 

Behavioral adjustment/ 
Self-perception of positive and 
negative behaviors 

28 

47.43 

47.50 

0.07 

1.38 

0.05 

.959 

.01 

Intellectual status/ 

Self-assessment of intellectual 
and academic abilities 

28 

45.43 

45.64 

0.21 

1.27 

0.17 

.867 

.03 

Physical appearance & attributes/ 
Self-appraisal of appearance, 
leadership, ability to express 
ideas 

28 

48.39 

49.14 

0.75 

1.36 

0.55 

.584 

.10 

Freedom from anxiety/ 

Positive emotional states, not 
bothered by sadness, worry, 
unpleasant mood states 

28 

46.36 

51.00 

4.64* 

1.71 

2.72 

.011 

.51 

Popularity/ 

Self-evaluation of social 
functioning, peer 
relationships 

28 

47.86 

50.79 

2.93 

1.72 

1.70 

TOO 

.32 

Happiness and satisfaction / 

Feelings of happiness and 
satisfaction with life 

28 

47.25 

51.18 

3.93** 

1.32 

2.97 

.006 

.56 


Note. T scores are standardized scores with a mean of 50 and a standard deviation of 10. On all Piers-Harris 2 scales except the 
total score, T scores from 40 to 55 are considered to be within the average range. On total score, T scores between 40 and 59 
comprise the average range. On all scales and total score, higher scores represent higher ratings of well-being. 


* p < .05; ** p < .01 
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How do teachers rate the classroom behavior of students at referral to LTL and after receiving 
services? 

The classroom behavior of a subset of students (N = 66) was rated by teachers using the Sutter- 
Eyberg Student Behavior Inventory-Revised (SESBI-R). Teachers rated the students’ behavior at 
referral to LTL and, during the same school year, after the student received mental health or family 
case management services at an LTL site. The therapist requested that the teacher complete the 
ratings; a total of 66 students were rated by their teachers at two time periods. Table 7 describes 
the students in this analysis. 


Table 7 

Characteristics of Elementary and Middle School Students With Pre- and Post- 
LTL Behavior Ratings on the SESBI-R, 2013-2014 


Characteristics 


so 

so 

II 

School Level 

n 

% 

Elementary School 

45 

68.2 

Middle School 

21 

31.8 

Race/Ethnicity 

Black or African American 

11 

16.7 

American Indian 

<5 

— 

Asian American 

<5 

— 

Hispanic/Latino 

41 

62.1 

White 

8 

12.1 

Two or More Races 

<5 

— 

Gender 

Female 

32 

48.5 

Male 

34 

51.5 

Service Provided 

ESOL (current) 

21 

31.8 

Spec. Educ. (current) 

15 

22.7 

FARMS (current) 

58 

87.9 


Overall, the demographic characteristics of the students with behavior ratings were similar to 
students who received LTL mental health and case management services (see Table 2). Table 8 
summarizes the main types of LTL services received by students who had pre- and post-LTL 
behavior ratings. It should be remembered that students may have received more than one type of 
service, and each service may have been received multiple times. Most of the students with Piers- 
Harris 2 data received family case management services and individual psychotherapy; about two 
thirds received family therapy 
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Table 8 

Services Received by Students With Pre- and Post-LTL Behavior Ratings on the SESBI-R, 2013-2014 
(N = 66) 


Type of mental health or case 
management service 

Number of students 
with service 

% of students with 
service 

Median times service 
received /participated 

Family case management 

63 

95.5 

6 

Family therapy 

44 

66.7 

3 

Individual therapy 

66 

100.0 

22.5 

Group therapy 

21 

31.8 

8 

* Number of service encounters includes 

services provided for students and family members; 

services were received during 


2013-2014. 


Table 9 shows the “Intensity” and “Problem” scale behavior ratings for elementary and middle 
school students at referral and after participation in LTL. The Intensity scale assesses the 
frequency with which the child displays problem behaviors. The Problem scale assesses the degree 
to which the rater (teacher) considers the behavior a problem. 


Table 9 

Classroom Behavior Ratings on the SESBI-R for Elementary and Middle School Students 
With LTL Services in 2013-2014 at Referral (Pre-Service) and After Participation (Post-Service) 




Pre-Service 

Post-Service 







T score 

T score 










Paired 

Mean 

Std. 



Effect size 

Elementary School 

n 

Mean 

Mean 

difference 

error 

t 

P 

" (dz) 

Intensity (frequency 
of problem) 

45 

51.42 

50.98 

0.44 

1.06 

0.42 

.68 

.06 

Problem (degree of 









problem) 

45 

51.00 

49.80 

1.20 

1.11 

1.08 

.29 

.16 

Middle School 

Intensity (frequency 
of problem) 

21 

48.95 

51.24 

-2.29 

1.90 

-1.21 

.24 

-.11 

Problem (degree of 
problem) 

21 

48.19 

50.10 

-1.91 

2.06 

-0.92 

.37 

-.08 


Note. T scores are standardized scores with a mean of 50 and a standard deviation of 10. High scores indicate a greater 
level of conduct-disordered behavior (Intensity) and greater impact on the rater (teacher) (Problem). On both scales, T 
scores greater than or equal to 60 are considered clinically significant; T scores below 60 are within normal range. 


Behavior ratings for elementary students decreased but were not statistically significantly different 
from the time of referral to after LTL services were received. Among middle school students, the 
ratings for the frequency of the behavior (Intensity) and the degree to which the behavior is a 
problem (Problem) increased over the course of the two ratings, but differences were not 
statistically significant (see Table 9). The mean ratings of both elementary and middle school 
groups, at referral and after LTL participation, were within the normal range (T scores < 60). 
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Ratings for some individual students were clinically significant; maximum T score was 75 for 
elementary students and 73 for middle school students. 

Was there a change in school attendance after LTL participation? 

The attendance rates and mean number of unexcused absences were measured for students who 
received mental health or case management services at LTL sites during the 2013-2014 school 
year and compared with their attendance measures the previous year. Table 10 summarizes 
attendance rates during 2013-2014 and the previous year for elementary and middle school 
students who received various LTL mental health and family case management services. 


Table 10 

Attendance Rate for Current and Previous Year by LTL Services Received for Students Receiving 


LTL Mental Health and Family Case Management Services During 2013-2014 


Attendance Rate (% days attended) 



2012- 

2013 

2013- 

2014 

Paired mean 

Std. 



Effect 

Elementary School 

N 

Mean 

Mean 

difference 

error 

t 

P 

size (d z ) 

Students whose families 









received case management 

503 

94.83 

94.79 

-0.05 

0.21 

0.64 

.818 

.04 

Students who received 









family therapy 

304 

94.63 

94.73 

0.10 

0.22 

0.46 

.645 

.04 

Students who received 









individual therapy 

369 

94.74 

94.68 

-0.05 

0.20 

0.26 

.794 

.02 

Students who participated 
in therapeutic group 

215 

95.01 

95.34 

0.37 

0.26 

1.31 

.193 

.13 

Students who participated 
in recreation group 

278 

94.96 

95.53 

0.57** 

0.22 

2.60 

.010 

.22 

Middle School 

Students whose families 









received case management 

240 

93.53 

92.66 

-0.86* 

0.43 

-2.02 

.044 

-.18 

Students who received 









family therapy 

123 

93.26 

92.27 

-0.99 

0.69 

-1.48 

.142 

-.19 

Students who received 









individual therapy 

144 

92.85 

91.85 

-1.00 

0.57 

-1.76 

.080 

-.21 

Students who participated 
in therapeutic group 

141 

93.61 

93.71 

0.10 

0.61 

0.17 

.867 

.02 

Students who participated 









in recreation group 

68 

95.70 

95.67 

-0.03 

0.40 

-0.08 

.933 

-.01 


Note. Only students who were enrolled in LTL for four months or more during 2013-2014 and were in Grades 1 through 8 
with attendance data for 2012-2013 and 2013-2014 were included in the analysis. 

*p< .05; ** p< .01 


Among elementary students who received mental health and/or family case management services 
at LTL sites, attendance was significantly higher during 2013-2014 than the previous year for 
students who participated in a recreation group (t = 2.60, p = .01). The difference was about 0.6 
percentage point, which is equal to about one day in a 180-day school year, and large enough to 
yield an effect size of practical significance ( d : = .22) (see Table 10). 

Among middle school students who received mental health and family case management services 
at LTL sites, one subgroup — students whose families received case management — showed a 
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statistically significant decrease in attendance rate from 2012-2013 to 2013-2014 (from 93.5 to 
92.7; t = -2.02, p < .05 [equivalent to about % of one day in a 180-day year]), but the effect size 
(d z = .18) did not reach the threshold for practical significance (see Table 10). 

Table 1 1 shows the mean number of unexcused absences for elementary and middle school 
students who received mental health and family case management services at LTL sites in 2013- 
2014, compared with the number of unexcused absences the previous year. 


Table 1 1 


Number of Unexcused Absences for Current and Previous Year by Services Received for 
Students Receiving LTL Mental Health and/or Family Case Management Services During 2013-2014 


Number of Unexcused Absences 

Elementary School 

N 

2012-2013 

Mean 

2013-2014 

Mean 

Mean 

difference 

Std. 

error 

t 

Effect 
p size ( d : ) 

Students whose families received 
case management 

506 

5.64 

5.95 

-0.31 

0.24 

-1.28 

.200 

-.08 

Students who received family 
therapy 

304 

5.79 

6.19 

-0.21 

0.33 

-0.64 

.525 

-.05 

Students who received individual 
therapy 

371 

5.83 

6.31 

-0.49 

0.29 

-1.68 

.093 

-.12 

Students who participated in 
therapeutic group 

215 

5.43 

5.80 

-0.37 

0.40 

-0.91 

.362 

-.09 

Students who participated in 
recreation group 

278 

4.85 

4.86 

-0.01 

0.31 

-0.02 

.981 

.00 

Middle School 

Students whose families received 
case management 

242 

7.25 

7.21 

0.03 

0.58 

0.05 

.957 

.00 

Students who received family 
therapy 

125 

7.31 

7.84 

-0.53 

0.88 

-0.60 

.550 

-.08 

Students who received individual 
therapy 

145 

8.06 

8.52 

-0.46 

0.85 

-0.54 

.588 

-.06 

Students who participated in 
therapeutic group 

142 

7.25 

6.32 

0.93 

0.82 

1.13 

.259 

.13 

Students who participated in 
recreation group 

68 

4.80 

3.49 

1.31* 

0.65 

2.00 

.050 

.34 


Note. Only students who were enrolled in LTL for four months or more during 2013-2014 and were in Grades 1 through 8 with 
attendance data for 2012-2013 and 2013-2014 were included in the analysis. 

* p< .05 


Among elementary students who received LTL mental health and/or family case management 
services, differences in the number of unexcused absences were not statistically significantly 
different from 2012-2013 to 2013-2014 for any of the service subgroups. Among middle school 
students, however, analyses of unexcused absences revealed that students who participated in a 
recreation group through LTL had statistically significantly fewer unexcused absences in 2013- 
2014 than in the previous year ( t = 2.0, p = .05); the effect size (d : = .34) indicated the difference 
was large enough to be of practical significance (see Table 1 1). 

Analyses of attendance measures also were conducted by grade level, and results are shown in 
Appendix E, Tables E-l (attendance rate) and E-2 (unexcused absences). Among elementary 
students who received services in LTL, attendance gains (percentage of days attended) were 
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greatest in Grade 3 (p < .10); Grade 5 students, however, had higher rates of unexcused absences 
the year following LTL services (p < .05). Findings by grade also were mixed for middle school 
students: Grade 6 students had lower attendance rates during 2013-2014 compared with the 
previous year (p < .05); however, students in Grades 7 and 8 reduced their rate of unexcused 
absences (p < .10). 

Was there a change in Work Habits grades for elementary students during the year LTL services 
were received? 

Two work habits that are graded on Grades 1 through 5 elementary report cards at each marking 
period were examined for elementary students who received services in LTL: Task Completion 
and Rules and Procedures. For each marking period, teachers assign one of three grades to each 
of these two learning skills: Not yet evident; Progressing; or Demonstrating. To examine change 
from Marking Period 1 to Marking Period 4, the scores for each student were classified 
dichotomously as “Demonstrating” or “Not demonstrating” (i.e., not yet evident or progressing). 
Table 12 shows the percentage of students demonstrating each of the skills at Marking Period 1 
and at Marking Period 4 by the LTL service received. Only students who were receiving LTL 
services during both Marking Period 1 and Marking Period 4 were included in the analysis. 


Table 12 


Percentage of Students Demonstrating Specified Work Habits in 1st and 4th Marking Periods: 
Students Receiving LTL Services During 2013-2014 by Services Received 




Number and Percentage Demonstrating Work 

H£ibit W-XT 

Work Habit 


Marking Period 1 

IVIGLN cmai 

Marking Period 4 test 

Task Completion 

N 

n 

% 

n 

% 

P 

Students whose families received 
case management 

333 

150 

45.0 

156 

46.8 

.627 

Students who received family 
therapy 

210 

85 

40.5 

100 

47.6 

.086 

Students who received individual 
therapy 

241 

96 

39.8 

107 

44.4 

.278 

Students who participated in 
therapeutic group 

104 

45 

43.3 

43 

41.3 

.851 

Students who participated in 
recreation group 

213 

107 

50.2 

111 

52.1 

.699 

Rules and Procedures 

Students whose families received 
case management 

332 

176 

53.0 

169 

50.9 

.515 

Students who received family 
therapy 

210 

107 

51.0 

103 

49.0 

.678 

Students who received individual 
therapy 

241 

111 

46.1 

109 

45.2 

.897 

Students who participated in 
therapeutic group 

104 

55 

52.9 

52 

50.0 

.678 

Students who participated in 
recreation group 

213 

119 

55.9 

121 

56.8 

.904 

Note. Only students who were receiving LTL services during 1 st and 4 th marking periods were 

included in analysis. 
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On Task Completion, slightly higher percentages of students in four of the five service groups 
were demonstrating the work habit in Marking Period 4 than in Marking Period 1 of the year they 
received LTL services, but the differences were not statistically significant. On Rules and 
Procedures, differences in percentages of students demonstrating the work habit were even 
smaller, and percentages for most of the service groups were lower in Marking Period 4 than in 
Marking Period 1. On average, the percentage of students demonstrating Task Completion and 
Rules and Procedures were only around half, even at Marking Period 4. 

Among middle school students who were suspended in 2013-2014 and received LTL services, 
what was the rate of repeat suspension in 2014-2015? 

Among ah middle school students who received mental health or family case management services 
in LTL during 2013-2014 (N = 762), the percentage who were suspended during that year was 
8.0% 0 = 61). 

The data were examined to detennine whether students who were suspended in 2013-2014 and 
who received LTL services, were also suspended in the year following (2014-2015). Table 13 
shows the numbers of students suspended in 2013-2014 who were also suspended the following 
year. 

Table 13 

Number and Percentage of Students Suspended During 2013-2014 and the Year Following: 

Middle School Students Receiving LTL Services During 2013-2014, 
All Students, and by Services Received 


Rules and Procedures 


Suspended in 
2013-2014 

Of students suspended in 
2013—2014, number 
suspended in 2014-2015 


N 

n 

% 

77 

All students who received family case 
management and mental health services 

762 

61 

8.0 

17 

Students who received family therapy 

150 

15 

10.0 

4 

Students who received individual therapy 

171 

27 

15.8 

6 

Students who participated in therapeutic 
group 

164 

14 

8.5 

5 

Students who participated in recreation 
group 

79 

<10 




Note. Students whose intake dates were April 20 14 or after were not included in analysis. 


Of the middle school students in LTL who were suspended during 2013-2014 (n = 61), only 17 
were suspended during the following school year. Although the numbers of students are small, 
when the suspensions are examined by services received, students who received individual therapy 
had the lowest percentage of repeat suspensions in the following year (6 of 27 = 22%). 

Evaluation question: After participating in Linkages to Learning services, to what extent did 
families show improvement in well-being and support of their student’s education, and 
increased community participation? 
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The demographic characteristics of family members who received LTL family case management 
or mental health services during the 2013-2014 school year are shown in Table 14. The 
demographic information reported in the table is based on LTL program records; race and ethnicity 
is recorded by the family case manager when a case is opened. More than three quarters of the 
family members who received LTL case management or mental health services in 2013-2014 were 
at elementary school LTL sites. Most of those who received LTL services were Hispanic/Latino 
(79%), and the largest race group served was white (57%), according to program records. 


Table 14 

Number and Percentage of Family Members 


Who Received LTL Case Management Services, 2013-2014 


Characteristics 


N= 1,025 

School level of LTL site 

n 

% 

Elementary School 

834 

81.4 

Middle School 

191 

18.6 

Race 

Black or African American 

149 

14.5 

American Indian 

4 

0.4 

Asian American 

13 

1.3 

White 

585 

57.1 

Information not captured 

274 

26.7 

Ethnicity 

Hispanic/Latino 

810 

79.0 

Non-Hispanic/Latino 

159 

15.5 

Information not captured 

56 

5.5 


Note. Race and ethnicity based on information collected by LTL program staff. 


A subset of family members receiving LTL services (N= 125) had ratings on the Massachusetts 
Family Self-Sufficiency Scales and Ladders Assessment Form (referred to in this report as the 
Massachusetts Scales) at intake into LTL and after receiving case management services for 
approximately six months. Families were rated on a live-point scale from “In-crisis” to “Thriving” 
in areas of employment, health, childcare, family development, housing, income management, 
transportation, nutrition, child’s education, adult education, and community participation. Table 
15 provides a summary of the ratings of these 125 families at the time of their referral to LTL and 
after receiving LTL services, with ratings combined into “Thriving/Stable/Safe” or “At-risk/In- 
crisis.” The percentage of families rated on all five levels of the Massachusetts Scales is shown in 
Appendix E, Table E-4. 
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Table 15 

Ratings of Family Members’ Self-Sufficiency at Referral to LTL and After Receiving LTL 
Services During 2013-2014 


Self-Sufficiency 
Rating a 


At Intake 

After 6 months 



Basic needs 

N 

Thriving/ 

Stable/Safe 

% 

At Risk/ 
In-crisis 

% 

Thriving/ 

Stable/Safe 

% 

At Risk/ 
In-crisis 

% 

Change 
in % 
Safe/+ 

McNemar 

Test 

P = 

Employment 

121 

43.0 

57.0 

58.7 

41.3 

15.7 

.001 

Health 

119 

75.6 

24.4 

88.2 

11.8 

2.6 

.004 

Childcare 

117 

77.8 

22.2 

91.5 

8.5 

13.7 

.000 

Family Dev. 

(safety, support) 

117 

83.8 

16.2 

93.2 

6.8 

9.4 

.007 

Housing 

120 

79.2 

20.8 

85.8 

14.2 

6.6 

.077 

Income 

Management 

116 

49.1 

50.9 

66.4 

33.6 

17.3 

.000 

Transportation 

120 

73.3 

26.7 

81.7 

18.3 

8.4 

.041 

Nutrition 

119 

78.2 

21.8 

92.4 

7.6 

14.2 

.000 

Education and community participation 

Education and 
Literacy: Child 

121 

84.3 

15.7 

91.7 

8.3 

7.4 

.049 

Education and 
Literacy: Adult 

121 

42.1 

57.9 

52.9 

47.1 

10.8 

.007 

Community 

Participation 

120 

63.3 

36.7 

80.8 

19.2 

17.5 

.000 


a Only families with at least two ratings are included. 


In the areas of employment, income management, and adult education and literacy, more than half 
of the family members with Massachusetts Scales ratings were rated “at-risk” or “in-crisis” at the 
time of intake to LTL. Other areas in which more than 20% of the family members were rated “at- 
risk” or “in-crisis” at referral were health, childcare, housing, transportation, nutrition, and 
community participation (Table 15). 

Do families show an increased capacity to meet basic needs? 

Table 15 also shows ratings of family members in areas of basic needs after six months in LTL. 
The mean number of months between the two ratings was 7.05 (median = 6.2). 

Improvement in family self-sufficiency ratings was evident in nearly all of the areas, with positive 
changes in the percentage of families rated “Safe” or better ranging from 3 to 17 percentage points. 
On seven of the eight areas of basic needs, the percentage of families who were rated “Safe,” 
“Stable,” or “Thriving” was significantly larger after LTL services compared to ratings at intake 
(McNemar test,/; < .05). The areas with the largest increase in the percentage of families rated 
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“Safe” or better between intake and after receiving LTL services — ten percentage points or more — 
were employment (43% at intake compared to 59% at 6 months), income management (49% 
compared to 66%), childcare (78% compared to 91%), and nutrition (78% compared to 92%) (see 
Table 15). Two of those areas — employment and income management — were areas with the 
highest percentage of “at-risk” or “in-crisis” ratings at the time of referral. 

Do families show increased support of student’s education at home and at school and increased 
community participation? 

Massachusetts Scales ratings. Table 15 shows the family members’ ratings in areas of education 
and community participation after receiving LTL services. Statistically significantly larger 
percentages of families were rated “Safe,” “Stable,” or “Thriving” in ratings after six months 
compared with ratings at intake to LTL in all three areas. The largest change was observed in 
community participation: 81% were rated “Safe,” “Stable,” or “Thriving” after receiving LTL 
services, compared with 63% at intake, a change of 18 percentage points. 

Parent/guardian surveys. In their responses to the end-of-year LTL survey, parents/guardians 
indicated their agreement that LTL helped them support their child’s education. Table 16 
summarizes responses to questions about these aspects of LTL participation. 

Table 16 

Level of Agreement With Statements on LTL Survey by Elementary and Middle School Parents: 


Support for Child’s Education 



Elementary School 


Middle School 



Responded Agree or 
Strongly Agree a 


Responded Agree or 
Strongly Agree a 

Survey statement 

N 

77 % 

N 

77 % 

LTL has helped me know how to 
communicate with teachers. 

286 

269 94.1 

92 

88 95.7 

LTL has helped me learn ways to help 
with schoolwork. 

291 

274 94.2 

90 

83 92.2 


a Parents responded to survey statements on a five-point scale: “Strongly Agree,” “Agree,” “Not Sure,” “Disagree,” 


and “Strongly Disagree.” The percentage who responded “Strongly Agree” or “Agree” are shown here. 


More than 90% of responding parents/guardians, both with students in elementary school and 
middle school, agreed that LTL helped them communicate with teachers and that LTL has helped 
them learn ways to help their children with schoolwork (see Table 16). 

Parents’/guardians’ responses to the LTL survey also indicated that their participation in LTL was 
helpful for their children and for their families in terms of their involvement and feelings toward 
school. Table 17 summarizes parents’/guardians’ responses. 
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Table 17 


Level of Agreement With Statements on LTL Survey by Elementary and Middle School Parents: 
Engagement in School by Student and Family 



Elementary School 


Middle School 



Responded Agree or 
Strongly Agree 3 


Responded Agree or 
Strongly Agree 3 

Survey Statement 

N 

n 

% 

N 

n 

% 

LTL has helped my child feel a part of the 
school. 

294 

281 

95.6 

92 

89 

96.7 

LTL has helped our family feel a part of 
the school. 

299 

289 

96.7 

91 

87 

95.6 

LTL has helped my child feel good about 
going to school. 

297 

281 

94.6 

86 

82 

95.3 

LTL has helped my child to get more 
involved in school activities. 

297 

283 

95.3 

57 

52 

91.2 


a Parents responded to survey statements on a five-point scale: Strongly Agree, Agree, Not Sure, Disagree, and 


Strongly Disagree. The percentage who responded “Strongly Agree” or “Agree” are shown here. 


Finally, although the data reporting participation in community education and development 
activities do not allow a determination of increased school engagement and support of students’ 
education, an examination of these data shows that LTL schools provided a range of programs and 
activities for parents/guardians, and a large number of parents/guardians participated in them. 
Table 18 shows attendance at activities supporting parents ’/guardians’ involvement in education 
during 2013-2014. 


Table 18 

Community Education and Development Activities Provided by LTL During 2013-2014 
School Year and Attendance by Families 


Program Type 

Examples of Activities 

Number of Sites 
Offering 
(25 reporting 3 ) 

Total Number 
of Sessions 
Offered by All 
Sites 

Total 

Attendance, 

All Sessions 

Adult education, 
skill development 

Description of MCPS resources, 
GED, legal presentation, 
computer classes, literacy 

18 

279 

782 

Adult ESOL 

classes 

Level I, Level II, Basic, 
intermediate conversation 

16 

675 

2,525 

Parent education 
and support 

Parent coffees, parenting skills, 
empowerment, positive discipline 

22 

400 

2,975 


3 Records were provided by 25 sites; two were combined programs (two sites serve two elementary schools each), and one 
middle school had limited staffing to provide programming for 2013-2014. 


Note. Activities listed for each program type are examples but are not all-inclusive. For some activities, the attendance 
counts were duplicative (participants were counted each time they attended an activity). 
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Summary 

Students who received mental health or family case management services at an LTL site showed 
improvement on some measures of well-being and school engagement. Specifically, on self- 
ratings of well-being before and after participation in LTL child/family therapy, elementary 
students improved on five of six scales plus an overall measure, and middle school students 
improved on two of five scales, plus an overall measure. Teacher ratings of students’ classroom 
behavior were not significantly different before and after students received child/family therapy 
through LTL 

Elementary report card grades on Work Habits were not statistically significantly different from 
Marking Period 1 to Marking Period 4 for students who were receiving mental health and/or family 
case management services in LTL during 2013-2014. Overall, only about half of the students 
receiving LTL services were demonstrating the Work Habits in Marking Period 4, a rate similar 
to Marking Period 1 . 

Attendance (percentage of days attended) for elementary students who participated in a recreation 
group through LTL was statistically significantly higher during the year of LTL participation 
compared to the previous year. For middle school students whose families received case 
management services, attendance rate was significantly lower during the LTL service year, but 
among middle school students who participated in recreation activities through LTL, the number 
of unexcused absences was significantly fewer compared with the previous year. An examination 
of suspension rates for students who were suspended during the year they received LTL services 
showed that nearly three quarters of them (72%) were not suspended during the following school 
year. 

Families who received case management services showed improvement on several areas of self- 
sufficiency. Ratings on health, nutrition, family development, income management, adult 
education and community participation were significantly higher after receiving LTL family case 
management services than at referral to LTL. In addition, in response to a survey at the end of the 
school year, large percentages of parents/guardians (more than 90%) agreed that LTL had helped 
them support their student’s education and feel part of the school and that LTL had helped their 
student feel part of school and become more involved in school activities. 


Discussion 

Assessing the impact of an initiative such as LTL, which provides a range of services in varying 
combinations and amounts according to the needs of the students or their families, is full of 
challenges, and several of these were discussed earlier in the Limitations section. With these 
challenges in mind, however, the study did provide some evidence of progress on two indicators 
of student well-being and engagement: ratings of self-concept and school attendance. On ratings 
of self-concept, effects for elementary students were more pronounced than effects for middle 
school students. On attendance, effects differed for elementary and middle school students. 
Several factors may contribute to these varied findings. First, the services provided for elementary 
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students and middle school students may have differed in nature and in amount. The mean number 
of encounters within some of the LTL services differed for elementary and middle school students 
(see Appendix F, Table F-l). In addition, the middle school LTL sites have fewer full-time staff 
and larger school communities, so some services are by necessity scarcer. Finally, some 
differences in observed impact between elementary and middle school students, such as well-being 
and attendance, may be related to developmental differences. It is possible that some behaviors 
may be more readily improved with elementary students than with middle school students, 
particularly when the intervention includes working with parents/guardians. 

Measures of family self-sufficiency before and after involvement in LTL services indicated that 
family members made progress in nearly all areas measured. Some areas, however, appear 
difficult to impact — specifically, employment, income management, and adult literacy. More than 
a third of the family members were still at risk or in crisis in these areas after LTL participation. 
These findings reinforce the understanding that families served by LTL bring challenging and 
complicated circumstances to the program. A unique strength of the program is its capacity to 
respond to multiple and changing needs of students and their families. 
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Recommendations 

1 . Utilize technology to strengthen the process for therapist and case manager to receive daily 
feedback about the attendance of students with whom they are working. Some evidence of 
improvements in attendance was observed for students receiving services in LTL sites, 
particularly for elementary students. Since school attendance is a key ingredient for academic 
success, closer, more real-time monitoring may strengthen LTL’s impact on attendance. 
Currently, LTL staff have access via parent/guardian consent to the MCPS student database; 
however, the database is not updated in real time. As such, outside of biweekly team meetings 
and direct contact with students and family members being served, LTL staff rely on 
notification from teachers, counselors, or other school-based staff. Workgroups are underway 
between MCDHHS and MCPS to develop a data-sharing process that will allow relevant 
information to be accessed more efficiently. 

2. Explore ways to increase systematic collaboration between LTL sendee providers and 
teachers to better understand specific issues and concerns of students who are not 
demonstrating skills associated with expected work habits. About one half of the students 
receiving mental health services and/or whose families were receiving case management at 
LTL sites were demonstrating the skills related to two report card work habits — Task 
Completion and Rules and Procedures — in Marking Period 4, a rate similar to Marking Period 
1 . 

3. Explore additional ways to support families in areas that appear most challenging to impact. 
Many areas of family self-sufficiency showed statistically significant improvement after 
family members received LTL case management services. Some areas — such as employment 
and income management — did show improvement during LTL participation but remain 
challenging for many families. Difficult problems such as these require additional resources 
and innovative approaches. LTL has been looking into new partnerships with Montgomery 
College to expand adult education opportunities offered on-site at LTL schools in hopes of 
improving opportunities for employment for parents/guardians. LTL should explore additional 
systemic partnerships to support outcomes in these areas. 

4. Institute more regular and consistent administration of LTL program assessments. A relatively 
small number of students who exited LTL mental health services during the 2013-2014 school 
year completed measures of well-being at referral and when exiting LTL services; likewise, 
few students had behavior ratings completed by teachers. Ratings of family self-sufficiency 
also were completed for a relatively small number of families who received case management 
services at LTL sites, although larger numbers have been collected in subsequent years. (The 
year reviewed in this study was only the second year for administering the family self- 
sufficiency scale. In order to obtain consistent baseline measures, the scale was administered 
only with families who were new to the program at intake, not with families who had been 
previously receiving LTL case management services.) Obtaining LTL measures for all 
students and families who receive LTL services is needed so that a more reliable summary of 
the status of clients’ well-being can be reported, more regular feedback can be provided to 
program staff, and additional infonnation is available for monitoring and adjusting services. 
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LTL is currently exploring the use of new electronic assessment measures in order to achieve 
this improvement. 

5. Increase means for feedback from students participating in LTL sendees. At the time of data 
collection for this study, needs surveys were administered to parents/guardians in schools with 
an LTL site, satisfaction surveys were administered each year to parents/guardians who 
received (or their children received) any LTL services, data scales were collected from students 
who received behavioral health services, family self-sufficiency scales were collected with 
families receiving case management services, and surveys for specific events and programs 
were collected. However, no consistent satisfaction surveys of students receiving LTL service 
were conducted across the initiative. It is recommended that two versions of surveys be 
administered at all LTL sites: (1) to a sample of all students in the school to identify ways to 
make services more accessible and target needs, interests, and ideas for groups and activities 
(to be coordinated with MCPS protocols for student survey administration); and (2) to students 
who have received any LTL service to ascertain what aspects of LTL have been most helpful 
and what additional services are needed. It should be noted that student needs and satisfaction 
surveys are now in place at LTL middle school sites, with plans underway to expand to higher 
elementary school grades in the future. 
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Appendix A: Logic Model of Linkages to Learning 


Service Area 

Outputs 

Short-term 

outcomes 

Long-term 

outcomes 


Student 

Well-being 

School health 

Social/ emotional/ 
behavioral health 

School consultation & 
prevention 

Primary Care and 
treatment (at LTL School- 
Based Health Centers) 


Students in SBHCs receive 
well visits 

Referred students are 
assessed and referred to 
appropriate services 

Referred students attend 
psychotherapy sessions 

LTL consults with teacher 
about needs/action plan for 
referred students 

Student attends 
recommended psychosocial 
skills groups 


Maximized 
attendance/ 
minimized tardiness & 
truancy 

Students report 
positive feelings of 
well-being and 
belonging in school 

Students express 
positive self-appraisal 




Students attend school 
consistently 


Students are actively 
involved in learning and in 
the school community 


Family Services 


Family needs assessment 

Family case management 
(providing/linking to 
concrete resources and 
benefits) 

Parent education (ESOL, 
adult education, parenting 
groups) 


Needs assessments 
conducted by LTL 

LTL completes case 
management for families 

Family follows resource 
recommendations 

Follow-ups by LTL re: 
recommendations 

Parent participation in 
adult education 

Families receive help with 
school system 


Families' increased ability 
to provide for basic needs 

Families are engaged in 
students' education at 
home 

Families attend school- 
wide events and 
conferences 


Increased school readiness 


Students are actively 
involved in their school 


Families are actively 
involved in children's 
education 


Community 


LTL conducts community 

Education and 


needs assessment 

Development 


School activities are 

Community needs 


offered 

assessment 


Students attend after- 

Out-of-school-time 


school activities 

programs targeted to 
community needs 

* 

Community and parent 
activities are offered 

Community-school 

partnerships 


Parents attend school- 

based activities 

Family engagement and 
leadership 


Partners involved with 

schools 


Students feel they 


Increased school 

belong in school 


readiness 

Schools are open to 
community 


Students are actively 
involved in their 
community. 

Families participate in 


Schools are engaged 

decision-making about 


with families and 

child's learning, LTL 


communities 

programming, and 
school-wide governance 


Families are 
engaged in students' 
education at home 

and at school 


Figure A-l. Logic model for Linkages to Learning (Office of Shared Accountability, MCPS Linkages to 
Learning Resource Team and Casey Family Programs, model developed 2013) 
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Appendix B 

Table B-l 

Services and Activities Offered at Linkages Sites 


Student well-being 

• Diagnostic assessment for social-emotional, behavioral concerns 

• Child/family/group psychotherapy 

• Classroom observation, teacher consultation 

• Homework club, tutoring, mentoring 

• Psychosocial skills development groups 

• Referral and case management for children with physical and medical issues 

• At SBHCs: Provision of services addressing mental health, preventive dental, and mental health needs 

Family services 

• Family needs assessments 

• Family case management (providing/linking to concrete resources and benefits, such as: 

assistance obtaining clothing, furniture, food, housing; assistance with legal/immigration; medical/dental referrals; 
employment needs; translation and transportation assistance; assistance accessing day care) 

• Parenting groups 

• Help navigating school system 

• Help navigating community resources 


Community education and development 

• Community needs assessment 

• ESOL classes, adult education, family literacy programs 

• Summer camp programs 

• Food/clothing/toy and book drives 

• Communitywide events 

• After-school and evening educational support programs 

• School system and community orientations for parents who are new Americans 
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Appendix C 



Elementary Schools: 

Fox Chapel ES 
Gartherstxrg ES S8HC 
RosemontES 
Summit Hal ESSBHC - 
Mental Health 
Washington Grove ES 
South Lake ES 

Middle Schools: 

Forest Oak MS 
Gartherstx/g MS 


Mental Health 


Association of 


Elementary Schools: 

Arcota ES 
Geonpan Forest ES 
Harmony His ESS8HC 
Ffc^iand ESSBHC 
Kemp MIES 
Sargent ShnverES 
VtersMd ESSBHC 
Weller Road ES S8HC 
wneaon woods to 

Middle Schools: 

loredermanMS 
Parkland MS 


Maryvale ES 


Elementary Schools: 

JtWm Leleck ES at Broad 
Acres S8HC 
GreencasOe ES 
•Montgomery Kncls ESI 
Pine Crest ES 

•New Hampshre Estates ES 
S8HC/OakViewES 
Rdrg Terrace ESS8HC 


Middle Schools: 
Eastern MS 

Slver Spmg Intematonal 


SCHOOL-BASED HEALTH CENTERS 

Children's Pediatricians a Associates. LLC On-sde 
nurse practitioners and health promotion, pedatnaans 
w*h 24/7 telephone coverage at 

> JoAnn Leleck ES at Broad Acres S8HC 
-GaMierstwgESSSHC” 

- Harmony Hills ES SBHC* 

- Holland ESSBHC 

- New Hampshire Estates ES S8Htr* 
-Rolling Terrace ESSBHC 
'Summit Hail ESSBHC" 

- Viers Mil ES S8HC 
'Weller Road ESSBHC 

‘Heath services also avarlatte to A/cofa ES 
"Health services also avatotVe to Washington Grove 
i Rosemont Elementary Schools 
"Wealth services also wafeMe to Oak WewES 




Figure C-l. Linkages to Learning collaborative structure 
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Appendix D 

Calculation of Effect Sizes 

Cohen (1988) provides these guidelines for interpretation: d = .20 is considered a small effect; 
d = .50 is considered a medium effect; d = .80 is considered a large effect. 

Cohen’s dz was used as an effect size estimate for the paired-samples differences. 

Cohen’s afwas calculated based on the paired f-test value (Rosenthal, 1991). 

d z = t / Vh 
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Appendix E 

Table E-l 

Attendance Rate for Current and Previous Y ear by Grade 
Students Receiving LTL Mental Health and Family Case Management Services, 2013-2014 

2013-2014 


2012-2013 (LTL Service) 


Elementary School 

N 

Mean 

Mean 

Paired 

Mean 

difference 

Std. 

error 

t 

P 

Effect 

size 

(f) 

Grade 1 

139 

94.24 

94.79 

0.56 

0.39 

1.44 

.154 

.17 

Grade 2 

212 

94.61 

94.72 

0.11 

0.28 

0.40 

.691 

.04 

Grade 3 

212 

94.73 

95.21 

0.48 

0.26 

1.85 

.065 

.18 

Grade 4 

238 

95.37 

95.35 

-0.02 

0.35 

-0.06 

.953 

.00 

Grade 5 

246 

95.29 

95.22 

-0.07 

0.21 

-0.35 

.729 

-.03 

Middle School 

Grade 6 

202 

95.65 

95.04 

-0.61* 

0.31 

-1.99 

.048 

-.20 

Grade 7 

185 

94.61 

94.39 

-0.23 

0.30 

-0.75 

.452 

-.08 

Grade 8 

158 

92.69 

92.66 

-0.04 

0.54 

-0.07 

.944 

.00 


Note. Students whose intake dates were April 20 14 or after were not included in analysis, and only students with attendance data 
for 20 1 2-20 1 3 and 20 1 3-20 1 4 were included. Previous year data for kindergarten was available only for students who attended 
MCPS pre-K. 


Table E-2 

Number of Unexcused Absences for Current and Previous Year by Grade 
for Students Receiving LTL Mental Health and Family Case Management Services, 2013-2014 

2013-2014 


2012-2013 (LTL Service) 


Elementary School 

N 

Mean 

Mean 

Paired 

Mean 

difference 

Std. 

error 

t 

P 

Effect 

size 

(df) 

Grade 1 

140 

5.51 

6.16 

-0.65 

0.47 

-1.38 

.169 

-.17 

Grade 2 

213 

4.95 

5.65 

-0.69 

0.38 

-1.80 

.073 

-.17 

Grade 3 

213 

5.61 

5.29 

0.33 

0.38 

0.85 

.395 

.08 

Grade 4 

240 

5.31 

5.23 

0.08 

0.37 

0.21 

.835 

-.02 

Grade 5 

248 

5.26 

5.93 

-0.67 

0.31 

-2.17 

.031 

-.20 

Middle School 

Grade 6 

203 

5.05 

5.11 

-0.06 

0.45 

-0.14 

.886 

-.01 

Grade 7 

187 

6.48 

5.40 

1.08 

0.58 

1.86 

.064 

.14 

Grade 8 

161 

8.30 

7.00 

1.30 

0.73 

1.78 

.077 

.14 


Note. Students whose intake dates were April 2014 or after were not included in analysis, and only students with attendance 
data for 2012-2013 and 2013-2014 were included. Previous year data for kindergarten were available only for students who 
attended MCPS pre-K. 
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Table E-3 

Ratings of Family Members’ Self-Sufficiency at Referral to LTL and After Receiving LTL Services 
During 2013-2014 





Thriving 

Stable 

Safe 

At-risk 

In-crisis 

Self-Sufficiency Rating a 

N 

% 

% 

% 

% 

% 

Basic needs 

Employment 

At intake 

121 

4.1 

11.5 

27.9 

43.4 

13.1 


After LTL services 


5.8 

13.2 

39.7 

35.5 

5.8 

Health 

At intake 

120 

19.0 

19.8 

36.4 

17.4 

7.4 


After LTL services 


18.3 

35.0 

35.0 

8.3 

3.3 

Childcare 

At referral 

118 

25.8 

40.8 

10.8 

15.0 

7.5 


After LTL services 


29.7 

44.1 

17.8 

8.5 

0.0 

Family Dev. 








(Safety, 

relationships. 

At intake 

118 

10.7 

28.9 

42.1 

15.7 

2.5 

support) 

After LTL services 


16.9 

36.4 

39.8 

6.8 

0.0 

Housing 

At intake 

121 

6.6 

24.8 

47.9 

18.2 

2.5 

After LTL services 


9.9 

28.9 

47.1 

11.6 

2.5 

Income 

At intake 

118 

1.7 

8.5 

39.0 

35.6 

15.3 

Management 

After LTL services 

0.0 

21.8 

44.5 

27.7 

5.9 

Transportation 

At intake 

120 

17.2 

26.2 

29.5 

22.1 

4.9 


After LTL services 


20.0 

32.5 

29.2 

14.2 

4.2 

Nutrition 

At intake 

119 

7.7 

11.8 

58.8 

21.0 

0.8 


After LTL services 


15.6 

12.3 

64.8 

7.4 

0.0 

Education and community participation 

Education and 
Literacy: 

At intake 

121 

17.4 

39.7 

27.3 

14.0 

1.7 

Child 

After LTL services 


22.1 

42.6 

27.0 

8.2 

0.0 

Education and 

At intake 

121 

7.4 

9.1 

25.6 

41.3 

16.5 

Literacy: Adult 

After LTL services 

8.2 

17.2 

27.0 

40.2 

7.4 

Community 

At intake 

120 

2.5 

12.5 

48.3 

33.3 

3.3 

Participation 

After LTL services 

5.7 

18.00 

56.6 

19.7 

0.0 

“Only families with at least two ratings are included. 
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Appendix F 


Table F-l 


Number of Elementary and Middle School Students Receiving Mental Health and Family Case 
Management Services and Mean Number of Encounters per Service 



Elementary 


Middle 

Type of Service 

n 

Mean 

number 

encounters 

n 

Mean 

number 

encounters 

Students received family therapy 

354 

5.2 

150 

3.2 

Students received individual therapy 

439 

17.2 

172 

16.1 

Students participated in therapeutic group 

256 

8.7 

165 

6.4 

Students participated in recreation 
group 

342 

10.5 

80 

3.6 

Student’s family received family case 
management/consultation 

635 

6.4 

295 

8.6 
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